Methods in epidemiology and public health: does practice match theory?

Epidemiology is widely perceived as a public health discipline within which methodology matters. Methods dominate educational curriculums and influential textbooks. 3 Epidemiological societies regularly feature methods sessions at their national and international meetings and, at least informally, the discipline recognises the methodologists who study the methods and the practitioners who use them. It follows that epidemiological methods, whether quantitative (for example, meta-analysis or logistic regression) or qualitative (for example, causal inference or narrative reviews), have a theoretical side and a more practical side. The theory behind a method and especially how that method should be practised “in theory” are discussed in textbooks and journal articles; methodological standards and guidelines are good examples. How a method is actually practised is found in its applications, in analytical studies, reviews, and other publications. The relation between these accounts—the extent to which practice matches theory—is the starting point of this paper. Three methods are of particular interest: meta-analysis, causal inference, and techniques for systematic narrative literature reviews. These methods were selected because serious concerns have been raised about how each is practised. These are also closely related methods, often appearing together in the same publication, typically a review paper, textbook chapter, or technical report within which a body of evidence is summarised and interpreted. 8 Nevertheless, each of these methods is distinct enough to have a recognisable theoretical literature and observable practice patterns. Perhaps the most important reason for looking carefully at the relation between how these methods are practised and how they are “supposed” to be practised is the key part they play in the assessment and interpretation of scientific evidence. These methods are central to the search for causal determinants of disease and for ways to use that knowledge to improve public health. Public health and clinical disciplines are joined by education, psychology and the social sciences in using these methods. Practitioners from many disciplines may therefore gain insight from this methodological inquiry, that begins at the juncture of theory and practice (do they match?) and uses contemporary philosophical concerns to illuminate what is uncovered there and to help lay out a course for the future. Do theory and practice match? Inquiries into the relation between any method as a method and the practice of that method are relatively uncommon. There are, however, a few recent examples in epidemiology. Very little mismatch was observed between the methodological accounts of case-control studies of cancer screening and their practice, perhaps because those who write the methods papers in this highly specialised area are also directly involved in designing investigations of screening programmes using the case-control approach. For other methods, the situation is not so neat. Considerable mismatch between methodological standards and actual practice was recently identified in clinical epidemiological studies of molecular and genetic factors. For the three more interpretative methods to be examined in this paper, inquiries into the relation between practice and theory reveal a complex and intriguing array of issues. I begin with meta-analysis.